The present invention relates to enhancement of multimedia content and more particularly to a system, method and apparatus for enhancing multimedia content with supplemental content.
Multimedia computer systems have become increasingly popular over the last several years due to their versatility and their interactive presentation style. A multimedia computer system can be defined as a computer system having a combination of video and audio outputs for presentation of audio-visual displays. A modern multimedia computer system typically includes one or more storage devices such as an optical drive, a CD-ROM, DVD (DVD-Video or DVD Audio etc), Laser Disc, Video Disc or Audio Disc, or a hard drive. Audio and video data are typically stored on one or more of these mass storage devices. In some file formats the audio and video are interleaved together in a single file, while in other formats the audio and video data are stored in different files, many times on different storage media. Audio and video data for a multimedia display may also be stored in separate computer systems that are networked together. In this instance, the computer system presenting the multimedia display would receive a portion of the necessary data from the other computer system via the network cabling.
Multimedia computer systems have become increasingly popular over the last several years due to their versatility and their interactive presentation style. A multimedia computer system can be defined as a computer system having a combination of video and audio outputs for presentation of audio-visual displays. A modern multimedia computer system typically includes one or more storage devices such as an optical drive, a CD-ROM, a hard drive, a videodisc, or an audio disc, and audio and video data are typically stored on one or more of these mass storage devices. In some file formats the audio and video are interleaved together in a single file, while in other formats the audio and video data are stored in different files, many times on different storage media. Audio and video data for a multimedia display may also be stored in separate computer systems that are networked together. In this instance, the computer system presenting the multimedia display would receive a portion of the necessary data from the other computer system via the network cabling.
Graphic images used in Windows multimedia applications can be created in either of two ways, these being bit-mapped images and vector-based images. Bit-mapped images comprise a plurality of picture elements (pixels) and are created by assigning a color to each pixel inside the image boundary. Most bit-mapped color images require one byte per pixel for storage, so large bit-mapped images create correspondingly large files. For example, a full-screen, 256-color image in 640-by-480-pixel VGA mode requires 307,200 bytes of storage, if the data is not compressed. Vector-based images are created by defining the end points (corners), thickness, color, pattern and curvature of lines and solid objects within an image. Thus, a vector-based image includes a definition that consists of a numerical representation of the coordinates of the object, referenced to a corner of the image.
Bit-mapped images are the most prevalent type of image storage format, and the most common bit-mapped-image file formats are as follows. A file format referred to as BMP is used for Windows bit-map files in 1-, 2-, 4-, 8-, and 24-bit color depths. BMP files contain a bit-map header that defines the size of the image, the number of color planes, the type of compression used (if any), and the palette used. The Windows DIB (device-independent bit-map) format is a variant of the BMP format that includes a color table defining the RGB (red green blue) values of the colors used. Other types of bit-map formats include the TIF (tagged image format file), the PCX (Zsoft Personal Computer Paintbrush Bitmap) file format, the GIF (graphics interchange file) format, and the TGA (Texas Instruments Graphic Architecture) file format.
The standard Windows format for bit-mapped images is a 256-color device-independent bit map (DIB) with a BMP (the Windows bit-mapped file format) or sometimes a DIB extension. The standard Windows format for vector-based images is referred to as WMF (Windows meta file).
Full-motion video implies that video images shown on the computer's screen simulate those of a television set with identical (30 frames-per-second) frame rates, and that these images are accompanied by high-quality stereo sound. A large amount of storage is required for high-resolution color images, not to mention a full-motion video sequence. For example, a single frame of NTSC video at 640-by-400-pixel resolution with 16-bit color requires 512K of data per frame. At 30 flames per second, over 15 Megabytes of data storage are required for each second of full motion video. Due to the large amount of storage required for full motion video, various types of video compression algorithms are used to reduce the amount of necessary storage. Video compression can be performed either in real-time, i.e., on the fly during video capture, or on the stored video file after the video data has been captured and stored on the media. In addition, different video compression methods exist for still graphic images and for full-motion video.
Examples of video data compression for still graphic images are RLE (run-length encoding) and JPEG (Joint Photographic Experts Group) compression. RLE is the standard compression method for Windows BMP and DIB files. The RLE compression method operates by testing for duplicated pixels in a single line of the bit map and stores the number of consecutive duplicate pixels rather than the data for the pixel itself. JPEG compression is a group of related standards that provide either lossless (no image quality degradation) or lossy (imperceptible to severe degradation) compression types. Although JPEG compression was designed for the compression of still images rather than video, several manufacturers supply JPEG compression adapter cards for motion video applications.
In contrast to compression algorithms for still images, most video compression algorithms are designed to compress full motion video. Video compression algorithms for motion video generally use a concept referred to as interframe compression, which involves storing only the differences between successive frames in the data file. Interframe compression begins by digitizing the entire image of a key frame. Successive frames are compared with the key frame, and only the differences between the digitized data from the key frame and from the successive frames are stored. Periodically, such as when new scenes are displayed, new key frames are digitized and stored, and subsequent comparisons begin from this new reference point. It is noted that interframe compression ratios are content-dependent, i.e., if the video clip being compressed includes many abrupt scene transitions from one image to another, the compression is less efficient. Examples of video compression which use an interframe compression technique are MPEG, DVI and Indeo, among others.
MPEG (Moving Pictures Experts Group) compression is a set of methods for compression and decompression of full motion video images that uses the interframe compression technique described above. The MPEG standard requires that sound be recorded simultaneously with the video data, and the video and audio data are interleaved in a single file to attempt to maintain the video and audio synchronized during playback. The audio data is typically compressed as well, and the MPEG standard specifies an audio compression method referred to as ADPCM (Adaptive Differential Pulse Code Modulation) for audio data.
A standard referred to as Digital Video Interactive (DVI) format developed by Intel Corporation is a compression and storage format for full-motion video and high-fidelity audio data. The DVI standard uses interframe compression techniques similar to that of the MPEG standard and uses ADPCM compression for audio data. The compression method used in DVI is referred to as RTV 2.0 (real time video), and this compression method is incorporated into Intel's AVK (audio/video kernel) software for its DVI product line. IBM has adopted DVI as the standard for displaying video for its Ultimedia product line. The DVI file format is based on the Intel i750 chipset and is supported through the Media Control Interface (MCI) for Windows. Microsoft and Intel jointly announced the creation of the DV MCI (digital video media control interface) command set for Windows 3.1 in 1992.
The Microsoft Audio Video Interleaved (AVI) format is a special compressed file structure format designed to enable video images and synchronized sound stored on CD-ROMs to be played on PCs with standard VGA displays and audio adapter cards. The AVI compression method uses an interframe method, i.e., the differences between successive frames are stored in a manner similar to the compression methods used in DVI and MPEG. The AVI format uses symmetrical software compression-decompression techniques, i.e., both compression and decompression are performed in real time. Thus AVI files can be created by recording video images and sound in AVI format from a VCR or television broadcast in real time, if enough free hard disk space is available.
As discussed above, such audio and video content is often stored on media such as CD-ROM or digital video disc (DVD). However, once a vendor has delivered such content to a customer, the vendor loses any practical control over the product. Even if the product is delivered under license rather than out right sale, it has traditionally been difficult to prevent a customer from copying the content or providing the content to any number of friends so that they might illegally copy the content.
The now familiar compact disk preserves information as a series of microscopic pits and smooth areas, oriented in concentric circular or helical tracks, on the otherwise smooth, planar surface of an annular disk. Recorded information is read from a compact disk by directing a focused laser beam along the recorded tracks, and detecting variations in the intensity of the laser beam along the recorded tracks, and detecting variations in the intensity of the laser beam as it encounters the microscopic pits and smooth areas on the disk. The coherence and relatively short wavelength of laser radiation enables large volumes of information to be written onto very small spaces of a recording medium.
Compact disks were first introduced in the music recording industry in 1982, and now account for 43% of all recorded music sales. In the United States alone, over three hundred million compact disks are sold annually, with a retail value of over three billion dollars, according to the Recording Industry Association of America. The most prevalent format for recording multimedia events onto such disks is Digital Video or Versatile Disk (DVD). The DVD is a read only format for recording a relatively large amount of high quality data. When delivered to a user, the disk is input into a CD-ROM player on a client device such as a computer. Software on the client device allows the DVD formatted data to be read.
Once the DVD disk has been manufactured the content is essentially fixed. The content that the user can access from the disk is limited to the content provided when the disk was manufactured. In order to update the information, a new disk must be created and delivered to the user. This is an expensive and inconvenient solution.
Thus there remains a need for a system for easily and efficiently updating content provided on a DVD-disk. Such a system would preferably allow update information to be delivered via a network such as the Internet. In addition, such a system would take advantage of software capabilities already present on the client device, and would importantly be able to function on the many different possible platforms of client devices, such as for example Macintosh, PC or a set top box.
Disc technologies that are re-writeable like a CD-RW or technologies that allow multiple sessions can be used for adding additional or updated content directly to the disc. Thus for multi-session discs, where the first session of the disc is write-once and additional sessions on the disc can be either write-one, or rewriteable, additional or updated content can be added to these additional sessions of the disc. This includes such technologies as the “Orange Book” specification for CD-ROM, including CD-PROM and Multimedia discs such a Dataplay.
Flash memory based and other similar memory technologies can be used for storing multimedia and additional or updated content as well. This includes IBM technology that uses a USB interface to coupled a personal computer to a storage device such as a “keychain” memory device.
The present invention advantageously addresses the above and other needs.